{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KjTHAV8FgEza"
      },
      "outputs": [],
      "source": [
        "# Copyright 2025 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UdL4uvQQs76x"
      },
      "source": [
        "# Veo 2 Editing\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_editing.ipynb\">\n",
        "      <img src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fvision%2Fgetting-started%2Fveo2_editing.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Run in Colab Enterprise\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/vision/getting-started/veo2_editing.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>    \n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_editing.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
        "\n",
        "<div style=\"clear: both;\"></div>\n",
        "\n",
        "<b>Share to:</b>\n",
        "\n",
        "<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_editing.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_editing.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_editing.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_editing.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_editing.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
        "</a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hcgplvn9SX2-"
      },
      "source": [
        "| | |\n",
        "|-|-|\n",
        "|Author(s) | [Katie Nguyen](https://github.com/katiemn) |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rDjAqcgigwdX"
      },
      "source": [
        "## Overview\n",
        "\n",
        "### Veo 2\n",
        "\n",
        "Veo 2 on Vertex AI brings Google's video generation capabilities to application developers. It's capable of creating videos with astonishing detail that simulate real-world physics across a wide range of visual styles.\n",
        "\n",
        "In this tutorial, you will learn how to use the Google Gen AI SDK for Python to interact with Veo 2 Editing to:\n",
        "- Remove content\n",
        "- Insert objects\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v2_iOv5uhXVg"
      },
      "source": [
        "## Get started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_vaNyOcDttya"
      },
      "source": [
        "### Install Google Gen AI SDK for Python"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "13Ly3sM8twuI"
      },
      "outputs": [],
      "source": [
        "%pip install --upgrade --quiet google-genai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GWYnCW0-h6HI"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "If you are running this notebook on Google Colab, run the following cell to authenticate your environment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bqz5LUG6h8fA"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LVrasKoriKZn"
      },
      "source": [
        "### Import libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oMQf_BkyiMgF"
      },
      "outputs": [],
      "source": [
        "import time\n",
        "\n",
        "from IPython.display import Image, Video, display\n",
        "from google import genai\n",
        "from google.genai import types"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yxBkUEqdiB1g"
      },
      "source": [
        "### Set Google Cloud project information and create client\n",
        "\n",
        "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
        "\n",
        "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "GtjPBmYHiEfx"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "PROJECT_ID = \"[your-project-id]\"  # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
        "\n",
        "if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
        "    PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
        "\n",
        "LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\")\n",
        "\n",
        "client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qD_bwA9hiMzL"
      },
      "source": [
        "### Define helper functions"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GUrEwbvFiPhJ"
      },
      "outputs": [],
      "source": [
        "def show_video(video):\n",
        "    if isinstance(video, str):\n",
        "        file_name = video.split(\"/\")[-1]\n",
        "        !gsutil cp {video} {file_name}\n",
        "        display(Video(file_name, embed=True, width=600, height=400))\n",
        "    else:\n",
        "        with open(\"sample.mp4\", \"wb\") as out_file:\n",
        "            out_file.write(video)\n",
        "        display(Video(\"sample.mp4\", embed=True, width=600, height=400))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2jaSOOadiUj6"
      },
      "source": [
        "### Load the video model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "APRfTklCiYR2"
      },
      "outputs": [],
      "source": [
        "video_model = \"veo-2.0-generate-preview\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8a2DFh8gi6gw"
      },
      "source": [
        "### Load the starting video\n",
        "\n",
        "Throughout this notebook, you'll edit the following video stored in Cloud Storage. Run the following cell to set the `video_gcs` variable you'll use in each request."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6wIBjxYyGYGu"
      },
      "outputs": [],
      "source": [
        "video_gcs = \"gs://cloud-samples-data/generative-ai/video/truck.mp4\"\n",
        "show_video(video_gcs)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jOAQvxAkP7C4"
      },
      "source": [
        "## Remove content with inpainting\n",
        "\n",
        "With inpainting removal, you can provide a mask to remove objects from a video. This can be done through static or dynamic inpainting."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aDaTx8WCidRG"
      },
      "source": [
        "### Static inpainting\n",
        "\n",
        "Static inpainting applies a mask to each frame of the video, allowing you to replace the masked area with new content.\n",
        "\n",
        "First, you'll download and display the mask image to be used in the request.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VKmYUdy9dlPp"
      },
      "outputs": [],
      "source": [
        "# Download the mask\n",
        "!wget https://storage.googleapis.com/cloud-samples-data/generative-ai/image/truck-inpainting-remove-static-mask.png\n",
        "\n",
        "# Display the mask\n",
        "mask_image = \"truck-inpainting-remove-static-mask.png\"\n",
        "display(Image(mask_image, height=400))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0dUp3Fuyj50J"
      },
      "source": [
        "In order to edit the video with this mask, specify the following:\n",
        "\n",
        "- **Prompt:** The new object you'd like to replace the masked content in each frame.\n",
        "- **Prompt enhancement:** The model offers the option to enhance your provided prompt. To utilize this feature, set `enhance_prompt` to True. A new, detailed prompt will be created from your original one to help generate higher quality videos that better adhere to your prompt's intent.\n",
        "\n",
        "**Safety:** All Veo videos include [SynthID](https://deepmind.google/technologies/synthid/)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tygfLLlWyTo_"
      },
      "outputs": [],
      "source": [
        "prompt = \"a mountain landscape\"  # @param {type: 'string'}\n",
        "\n",
        "operation = client.models.generate_videos(\n",
        "    model=video_model,\n",
        "    source=types.GenerateVideosSource(\n",
        "        prompt=prompt, video=types.Video(uri=video_gcs, mime_type=\"video/mp4\")\n",
        "    ),\n",
        "    config=types.GenerateVideosConfig(\n",
        "        mask=types.VideoGenerationMask(\n",
        "            image=types.Image.from_file(location=mask_image),\n",
        "            mask_mode=types.VideoGenerationMaskMode.REMOVE_STATIC,\n",
        "        ),\n",
        "        enhance_prompt=True,\n",
        "    ),\n",
        ")\n",
        "\n",
        "while not operation.done:\n",
        "    time.sleep(10)\n",
        "    operation = client.operations.get(operation)\n",
        "    print(operation)\n",
        "\n",
        "if operation.response:\n",
        "    show_video(operation.result.generated_videos[0].video.video_bytes)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fOBoqhdcXs4e"
      },
      "source": [
        "### Dynamic inpainting\n",
        "Dynamic inpainting uses a mask to select and remove an object from the first frame of a video. Only one object can be removed at a time.\n",
        "\n",
        "This example uses a video and mask image downloaded from Cloud Storage."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "urbTLTRxf2E3"
      },
      "outputs": [],
      "source": [
        "# Download the mask\n",
        "!wget https://storage.googleapis.com/cloud-samples-data/generative-ai/image/truck-inpainting-dynamic-mask.png\n",
        "\n",
        "# Display the mask\n",
        "mask_image = \"truck-inpainting-dynamic-mask.png\"\n",
        "display(Image(mask_image, height=400))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r-0qllN_g4c-"
      },
      "source": [
        "Since you're removing content from a video you don't need to specify a prompt."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YoJ4ucV_YDhQ"
      },
      "outputs": [],
      "source": [
        "operation = client.models.generate_videos(\n",
        "    model=video_model,\n",
        "    source=types.GenerateVideosSource(\n",
        "        video=types.Video(uri=video_gcs, mime_type=\"video/mp4\")\n",
        "    ),\n",
        "    config=types.GenerateVideosConfig(\n",
        "        mask=types.VideoGenerationMask(\n",
        "            image=types.Image.from_file(location=mask_image),\n",
        "            mask_mode=types.VideoGenerationMaskMode.REMOVE,\n",
        "        ),\n",
        "        enhance_prompt=True,\n",
        "    ),\n",
        ")\n",
        "\n",
        "while not operation.done:\n",
        "    time.sleep(10)\n",
        "    operation = client.operations.get(operation)\n",
        "    print(operation)\n",
        "\n",
        "if operation.response:\n",
        "    show_video(operation.result.generated_videos[0].video.video_bytes)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YysYLyiVj8Zd"
      },
      "source": [
        "## Add objects with inpainting\n",
        "\n",
        "With inpainting insertion, you can use a mask to add objects to a video.\n",
        "\n",
        "This example uses the same video and a mask image from Cloud Storage."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ekqkFIekgvH8"
      },
      "outputs": [],
      "source": [
        "# fmt: off\n",
        "mask_gcs = \"gs://cloud-samples-data/generative-ai/image/truck-inpainting-dynamic-mask.png\"\n",
        "# fmt: on\n",
        "\n",
        "# Display mask image\n",
        "file_name = mask_gcs.split(\"/\")[-1]\n",
        "!gsutil cp {mask_gcs} {file_name}\n",
        "display(Image(file_name, height=400))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O0CfYJlDhykP"
      },
      "source": [
        "**Output file location:** The generated video will be displayed below and saved to a Cloud Storage bucket. Specify the output path in the `output_gcs` variable."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "EuzdAORfhA7n"
      },
      "outputs": [],
      "source": [
        "output_gcs = \"gs://[your-bucket-path]\"  # @param {type: 'string'}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qq6DK5GOk-WU"
      },
      "source": [
        "This request will be similar to the others, except you'll reference the mask image from Cloud Storage directly, rather than using a locally downloaded image. To add content in this request specify the following:\n",
        "  - **Prompt:** The object you'd like added in the masked area.\n",
        "  - **Mask GCS:** The Cloud Storage location of your image mask.\n",
        "  - **Mask mime type:** Select either `image/png` or `image/jpeg`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5gNxNj9N2R4o"
      },
      "outputs": [],
      "source": [
        "prompt = \"sheep\"  # @param {type: 'string'}\n",
        "\n",
        "operation = client.models.generate_videos(\n",
        "    model=video_model,\n",
        "    source=types.GenerateVideosSource(\n",
        "        prompt=prompt, video=types.Video(uri=video_gcs, mime_type=\"video/mp4\")\n",
        "    ),\n",
        "    config=types.GenerateVideosConfig(\n",
        "        mask=types.VideoGenerationMask(\n",
        "            image=types.Image(\n",
        "                gcs_uri=mask_gcs,\n",
        "                mime_type=\"image/png\",\n",
        "            ),\n",
        "            mask_mode=types.VideoGenerationMaskMode.INSERT,\n",
        "        ),\n",
        "        output_gcs_uri=output_gcs,\n",
        "        enhance_prompt=True,\n",
        "    ),\n",
        ")\n",
        "\n",
        "while not operation.done:\n",
        "    time.sleep(10)\n",
        "    operation = client.operations.get(operation)\n",
        "    print(operation)\n",
        "\n",
        "if operation.response:\n",
        "    show_video(operation.result.generated_videos[0].video.uri)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "veo2_editing.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
